A PROJECT SUBMITTED TO THE FACULTY OF THE GRADUATE SCHOOL OF THE UNIVERSITY OF MINNESOTA DULUTH BY Xingguo Li IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN APPLIED AND COMPUTATIONAL MATHEMATICS

In this project, we discuss high-dimensional regression, where the dimension of the multivariate distribution is larger than the sample size, i.e. d n. With the assumption of sparse structure of the underlying multivariate distribution, we take the advantage of the `1 regularized method for parameter estimation. There are two major problems that will be discussed in this project: (1) a family of Least Absolute Shrinkage and Selection Operator (Lasso) type high-dimensional sparse regression methods (FLARE), including Least Absolute Deviation (LAD) Lasso, Square Root (SQRT) Lasso, `q norm loss Lasso and Dantzig selector; (2) precision matrix estimation with the Constrained `1 Minimization Approach to Sparse Precision Matrix Estimation (CLIME) and TuningInsensitive Approach for Optimally Estimating Gaussian Graphical Models (TIGER) respectively. We apply several different numerical algorithms to solve the proposed problems above. They are mainly Alternating Direction Method of Multipliers (ADMM) with convergence rate O(1/t) and the Monotone Fast Iterative Soft-thresholding Algorithm (MFISTA) with convergence rate O(1/t2), where t is the number of iterations. For CLIME and TIGER, we adopt a hybrid ADMM (HADM) which combines coordinate descent for fast convergence and linearization for stable estimation. For FLARE, we adopt both ADMM and MFISTA for LAD Lasso and SWRT Lasso combining dual smoothing, and we adopt ADMM for `q Lasso and Dantzig selector due to the different constraint of Dantzig selector and special non-smoothness form of `q Lasso. Especially for `q Lasso, we apply two root finding procedures to obtain its numerical estimation. Empirically, we conduct numerical experiments on both simulated and real data sets to illustrate the efficiency of our proposed method.